1. Field of the Invention
This invention generally relates to digital information communication and storage, and more particularly to a system and method for accelerating the uploading of data from a client to a network-connected remote service provider.
2. Description of the Related Art
As noted in Wikipedia, a proxy server is a server (a computer system executing an application program) that acts as an intermediary for requests from clients seeking resources from other servers. A client connects to the proxy server, requesting some service, such as a file, connection, web page, or other resource, available from a different server. The proxy server evaluates the request according to its filtering rules. For example, it may filter traffic by IP address or protocol. If the request is validated by the filter, the proxy provides the resource by connecting to the relevant server and requesting the service on behalf of the client. A proxy server may optionally alter the client's request or the server's response, and sometimes it may serve the request without contacting the specified server. In this case, it ‘caches’ responses from the remote server, and returns subsequent requests for the same content directly.
A proxy server may be used to speed up access to resources through caching. For example, web proxies are commonly used to cache web pages from a web server. A proxy server can be placed in the user's local computer or at various points between the user and the destination servers on the Internet. A caching proxy server accelerates service requests by retrieving content saved from a previous request made by the same client or even other clients. Caching proxies keep local copies of frequently requested resources, allowing large organizations to significantly reduce their upstream bandwidth usage and cost, while significantly increasing performance. Most ISPs and large businesses have a caching proxy.
A proxy that focuses on World Wide Web traffic is called a “web proxy”. The most common use of a web proxy is to serve as a web cache. Most proxy programs provide a means to deny access to URLs specified in a blacklist, thus providing content filtering. This is often used in a corporate, educational or library environment, and anywhere else where content filtering is desired. Some web proxies reformat web pages for a specific purpose or audience, such as for cell phones and PDAs.
While a web proxy is useful in speeding inbound web content, it does not address the sending of outbound traffic from a client to a server. The lack of an outbound web proxy is compounded by the fact that most cable providers configure their inbound bandwidth (to the client) to be significantly broader than the outbound bandwidth from the client. This fact is evident to any residential client that has attempted to upload large amounts of data (e.g., a photo album) to a service provider.
If security was not an issue, then an intercepting proxy could perform some of the functions of an outbound web proxy. An intercepting proxy combines a proxy server with a gateway or router. Connections made by client browsers through the gateway are diverted to the proxy without client-side configuration (or often knowledge). Connections may also be diverted from a SOCKS server or other circuit-level proxies.
Intercepting proxies are also commonly referred to as “transparent” proxies, or “forced” proxies, presumably because the existence of the proxy is transparent to the user, or the user is forced to use the proxy regardless of local settings. Intercepting proxies are commonly used in businesses to prevent avoidance of acceptable use policy, and to ease administrative burden, since no client browser configuration is required. This second reason however is mitigated by features such as Active Directory group policy, or DHCP and automatic proxy detection. Intercepting proxies are also commonly used by ISPs in some countries to save upstream bandwidth and improve customer response times by caching.
The diversion/interception of a TCP connection creates several issues. First, the original destination IP and port must somehow be communicated to the proxy. This is not always possible (e.g., where the gateway and proxy reside on different hosts). There is a class of cross site attacks which depend on certain behavior of intercepting proxies that do not check or have access to information about the original (intercepted) destination. This problem can be resolved by using an integrated packet-level and application level appliance or software which is then able to communicate this information between the packet handler and the proxy.
Intercepting also creates problems for HTTP authentication, especially connection-oriented authentication such as NTLM, since the client browser believes it is talking to a server rather than a proxy. This can cause problems where an intercepting proxy requires authentication. Then, the user connects to a site which also requires authentication. Finally, intercepting connections can cause problems for HTTP caches, since some requests and responses become uncacheble by a shared cache.
While intercepting connections are generally discouraged, due to the simplicity of deploying such systems, they are in widespread use. It is often possible to detect the use of an intercepting proxy server by comparing the client's external IP address to the address seen by an external web server, or sometimes by examining the HTTP headers received by a server.
Some of the functions of an outbound web proxy are performed in wide area networks (WANs), where security is less of a concern. WAN acceleration is a usage-based service with some pre-population capabilities. However, it is only used in the context of a private network based on the typical traffic pattern. A pre-population feature is required to manually setup a replication between the WAN accelerators and the server with the data. Thus, data is only accelerated to a limited number of modified servers.
It would be advantageous if data could be more quickly and securely uploaded from a client to a service provider server, without having to use specially modified servers.